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Domain Analysis 

• First step was to convene panel of content experts 

• Identify the content that represented best practices in 
teaching and learning 

• Deep conceptual understanding is promoted when 
learning is organized around “big picture” ideas 

• Content for domain organized and prioritized in increasing 
specificity starting with the big ideas of the discipline 

• Identify the skills to be developed in the course 

• Key goal was to move away from a domain that 
emphasized content facts 
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Domain Analysis 

Inputs to Domain analysis 

• Current AP course descriptions 

• National and state standards 

• Latest research on student learning and assessment 

• College Curriculum Study 

• Importance ratings for content and skill 

• Data on their teaching practices and course format 

• Uploaded documents (e.g., syllabus, assessments) 

• Expert judgment of panelists to make final decisions 
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Domain Model 


• Claims: statements about students’ knowledge and skill 
proficiencies 

• Summative claims represent all of the content and skills that 
should be acquired after learning a particular domain 

• Formative claims represent a subset of the content and skills 
intended to guide teaching and assess progress 

• Evidence: actual student work that is required to support 
the claims; must be observable. 

• Articulation of claims and evidence useful because it goes 
beyond a simple listing content and skill 
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Writing ciaims: Guideiines 


• Claims start with “The student can...” 

• To reinforce that fact that claims are made about what 
students should know or be able to do 

• Each claim required a verb or verb phrase that 
represented the skill involved in the claim 

• For science, these verbs came directly from the list of 
skills that were considered important in the domain 

• Each claim required a piece of content from 
the domain analysis 
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Writing ciaims: Content and Skiii Pairing 


• What content and skill pairings are most 
appropriate or ideal? 

• All possible pairings of content and skills were not 
appropriate or feasible given the learning goals and 
constraints of the summative exam 

• Determine and reach consensus on the most ideal 
content and skill pairings 

• Examples of Ideal pairings: promote conceptual 
understanding, required the student to go beyond 
simple rules, or promoted depth of understanding 
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Writing ciaims: Grain Size 


• At what level of specificity or grain size should 
the claim and evidence pairs be written? 

• General guideline was that grain size of the claim 
should be such that it can be supported by a 
manageable amount of observable evidence 

• Does claim provoke the question: “What does this 
mean?” Too general, difficult to articulate evidence 

• Is claim too specific? Oniy evidence that can be 
articulated is a restatement of the ciaim 
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Writing claims: Proficiency level 


• What is the target proficiency level of the claim and 
evidence pair? 

• Claims should represent summative expectations about 
what students should know and be able to do at the end 
of AP course 

• Further defined as any claim one would want to make 
about an AP student at the end of the course who 
deserves college credit 

• 84 to 119 claims written depending on subject 
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Defining Evidence 


• Evidence started with the phrase “The work is 
characterized by...” to reinforce the fact that 
evidence includes characteristics of work that is 
produced by students 

• Evidence should include only nouns and 
adjectives to emphasize that evidence must be 
concrete and observable 

• Evidence cannot include reference to the student 
or the task 
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Defining evidence: Skiii definitions 


• Key component of the process was defining skiiis in terms of 
observabie evidence 

• Appiv mathematical routines to quantities that describe natural 
phenomena 

• Correctness of equation and formulas 

• Correctness of application of mathematical routine 

• Reasonableness of solution given context 

• Definitions are helpful because offers a way to represent the 
observable characteristics of important skills 

• Ensures consistency between the evidence associated with 
claims that evoke the same skill but address different content 
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Claim and Evidence example 


• Claim: The student can apply mathematics in which they 
evaluate the reasonableness of quantities found in 
stoichiometric calculations. 

• Evidence: 

• Correctness of chemical equation 

• Correctness of chemical formulas 

• Correctness of application of mathematical routine 

• Correctness of coefficients interpreted as mole ratios 

• Reasonableness of solution as it related to mole ratio and differing 
molar masses 
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Challenges & Future Research 


• Iterative nature of work 

(1) Skill definitions initially created were preliminary 

( 2 ) Process of writing claims and evidence was itself iterative 

(3) Strained project timeline 

• Steep learning curve - difficult to think in terms of 
observable evidence 

• Defining appropriate grain size at which to write 
claims and evidence 
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Advantages of ECD 

• Knowledge about how deep conceptual understanding 
and complex reasoning skills are acquired and evidenced 
in specific subject areas is still evolving 

• Articulating claims and evidence for these disciplines helped 
further understanding within the disciplines themselves 

• Having evidence of the skill integrated with the content 
provides teachers with more guidance for designing 
instruction that develops content and skill in the context of 
each other. 

• No guesswork on the part of the item writer about what is 
valued in the domain or what characteristics of student 
works are required for evidence. 
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Thank You! 


• Researchers are encouraged to freely express their 
professional judgment. Therefore, points of view or 
opinions stated in College Board presentations do not 
necessarily represent official College Board position 
or policy. 

• Access this presentation online at 
http://profess onals.collegeboard.com/data-reports- 
research/cb/presentations 

• Please forward any questions, comments, and 
suggestions to: 

Maureen Ewing at: mewing(5)collegeboard.org 
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